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METHOD AND SYSTEM FOR PRODUCT OPTIMIZATION 
CROSS-REFERENCE TO RELATED APPLICATION 

This application claims the benefit of U.S. 
provisional application Serial No. 60/275,498, filed March 
13, 2001, and entitled "Six Sigma". 

BACKGROUND OF THE INVENTION 
1. Field of the Invention 

The present invention relates generally to six- 
sigma (6a) optimization models and, more particularly, to a 
six-sigma model which incorporates the voice of the customer 
and product design into the overall optimization process. 
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2 . Background Art 

In the context of manufacturing optimization, 
sigma (a) is a statistical unit of measure that reflects a 
level of manufacturing capability. Table 1 contains an 
exponential scale for quantifying manufacturing capability 
based on a conventional 6a philosophy. 

Table 1 



a 


DEFECTS PER MILLION 
OPPORTUNITIES (DPMO) 


% SUCCESS 


6 


3.4 


99.99966 


5 


233 


99.977 


4 


6,210 


99 .38 


3 


66, 807 


93 .32 


2 


308,537 


69.2 
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Notably, a slight increase in sigma level results 
in an exponential reduction in the number of defects per 
million manufacturing opportunities. In accord with 
traditional notions of 6a manufacturing capability, a 
5 manufacturer does not attain a "best-in-class " (BIC) status 
until a 6a capability level (i.e., < 3.4 DPMO) is attained. 

In many cases, a manufacturer's ability to 
transcend the 5a level of success is hampered by the fact 
that 70-80% of product defects are a direct or indirect 
10 result of designed-in quality problems (i.e., problems that 

H cannot be resolved by an increase in manufacturing 

Q 

q capabilities alone) . Accordingly, to attain a 6a level of 

4 s success, the optimization methodology should include 

Iff elements of product design. Most models for 6a 

15 optimization, however, do not incorporate the necessary 

s design aspect of optimization. Conventional 6a models 

p involve the steps of measuring existing manufacturing 

y. capabilities, analyzing mechanical opportunities for 

^ decreasing variability associated with the existing 

ffj 20 manufacturing process, improving the capabilities of the 

manufacturing process by adopting the most cost-effective 
improvement opportunities, and controlling the manufacturing 
process to minimize variability. 

Furthermore, although a few 6a models have 
25 attempted to incorporate product design into product 

optimization, these models seek to enhance the robustness of 
product design without incorporating a detailed analysis of 
the voice of the customer. Although existing 6a design 
optimization models including design optimization may 
30 improve product robustness, they do not analytically focus 

on opportunities for increasing product robustness which are 
most critical to customer satisfaction. Put another way, 
conventional 6a design optimization models arbitrarily 



- 2 - 



increase product robustness where possible without 
quantitatively and qualitatively weighing whether doing so 
will ultimately increase customer satisfaction beyond a 
negligible level. 

5 

SUMMARY OF THE I WENT ION 

One object of the present invention is to provide 
a method and system for product optimization which is based 
10 on an analytical methodology for translating the "voice of 

the customer" into aspects of the overall product that are 
critical to customer satisfaction (i.e., CTS aspects). This 
object is advantageous because it focuses optimization 
W efforts on product aspects that, upon improvement, will 

m 15 yield a higher level of customer satisfaction. 

u? Another object of the present invention is to 

D provide a methodology for quantitatively and/or 

J qualitatively characterizing contributing factors to each 

P CTS aspect in terms of their design level and noise control 

tj 2 0 factors (i.e., nominal design values). In accordance with a 

preferred embodiment of the present invention, the 
contributing factors (y) are characterized in terms of a 
transfer function y=f(x,n). This object is advantageous 
because it translates CTS aspects into engineering/criteria 
25 that can be optimized during product design and development. 

Yet another object of the present invention is to 
provide a system and methodology for expressing variability 
oy experienced in each contributing factory y based on the 
transfer function and nominal and variability values for 
30 each control factor x and noise factor n. This object is 

advantageous because it facilitates an iterative process of 
product design optimization wherein nominal design and 
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variability values for control factors x are adjusted to 
attain target values for y and ay. 

To meet these objects, features and advantages as 
well as additional objects, features and advantages, 
5 preferred and alternate embodiments of a method for product 

optimization are provided which include identifying at least 
one aspect of a product that is critical to customer 
satisfaction and a target value for at least one aspect, 
characterizing the at least one aspect in terms of at least 
10 one contributing factor, characterizing the at least one 

\± contributing factor in terms of at least one control factor, 

2 and adjusting nominal design values for the at least one 

J control factor such that variability in the at least one 

W contributing factor is minimized and the target value for 

pi 

yy 15 the at least one aspect is attained. 

y1 The at least one contributing factor may be 

p characterized in terms of the at least one control factor 

r based on a transfer function y=f (x) . The at least one 

H contributing factor may be additionally characterized in 

~; 20 terms of at least one contributing noise factor and a 

transfer function y=f(x,n). 

The at least one aspect and target value for the 

at least one aspect may be identified based on consumer 

insight . 

25 The method may further comprise identifying a 

target value for the at least one aspect in an aged 
condition, and minimizing variability in the at least one 
contributing factor such that the target value for the at 
least one aspect in the aged condition is attained. 

30 The method may additionally comprise minimizing 

variability in the at least one contributing factor by 
reducing variability in the at least one control factor. 
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Variability in the at least one contributing 
factor may be reduced to a 6a level. 

Additionally, preferred and alternate embodiments 
of a computer- implemented system for facilitating product 
optimization are provided with include a computer system 
configured to receive input defining a transfer function 
characterizing a contributing factor to a product aspect 
that is critical to customer satisfaction in terms of at 
least one control factor for the contributing factor, to 
receive input defining a nominal design value and a 
variability value for the at least one control factor, and 
to output a mean and variability value for the contributing 
factor based on the transfer function and the nominal and 
variability values for the at least one control factor. 

A transfer function y=f (x,n) may be implemented to 
characterize the contributing factor y in terms of the at 
least one control factor x and at least on noise factor n. 
The system may be additionally configured to receive input 
defining a nominal value and a variability value for each at 
least one noise factor n and output a nominal and a 
variability value for the contributing factor based on the 
transfer function, the nominal design value and variability 
value for the at least one control factor and the nominal 
value and the variability value for the at least one noise 
factor . 

Variability in the contributing factor may be 
reduced in response to input adjusting the nominal design 
value (s) for the at least one control factor. 

Variability in the contributing factor may be 
reduced in response to input reducing the variability 
value (s) for the at least one control factor. 

Variability of the contributing factor may be 
reduced to a 6a level. 



The above objects and other objects, features, and 
advantages of the present invention are readily apparent 
from the following detailed description of the best mode for 
carrying out the invention when taken in connection with the 
5 accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is a block flow diagram illustrating an 
10 overview of a preferred method for implementing the present 

invention; 

Figure 2 is a block flow diagram illustrating an 
overview of the defining aspect of the present invention; 

Figure 3 illustrates an example brand bull's eye 
15 for organizing consumer insight in accord with the present 

invention; 

Figure 4 illustrates an example brand profile 
including a product attribute leadership strategy (PALS) in 
accord with the present invention; 
20 Figure 5 is a quadrant chart for relating percent 

high customer satisfaction with relative leverage in accord 
with the present invention; 

Figure 6 is a block flow diagram illustrating an 
overview product benchmarking in accord with the present 
25 invention; 

Figure 7 illustrates a gap analysis associated 
with product benchmarking in accord with the present 
invention; 

Figure 8 illustrates a Kano analysis for analyzing 
30 data collected in accord with the defined aspect of the 

present invention; 
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Figure 9 illustrates schematic diagram (i.e., 
"house of quality") for organizing QFD data in accord with 
the present invention; 

Figure 10 illustrates a scorecard for assessing 
and summarizing CTS aspects in accord with the present 
invention; 

Figure 11 is a block flow diagram illustrating an 

overview of the characterized aspect of the present 

inventions- 
Figure 12a illustrates a function map template for 

characterizing CTS aspects in accord with the present 

invention; 

Figure 12b illustrates various methodologies and 

tools that may be implemented in generating transfer 

functions in accord with the present inventions- 
Figure 13 is a block flow diagram illustrating a 

preferred method for creating a function structure in accord 

with the present inventions- 
Figure 14 illustrates an example geometric 

function structure for an automotive regenerative braking 

system in accord with the present inventions- 
Figure 15 illustrates a transfer function format 

and example transfer function in accord with the present 

inventions- 
Figure 16 is a block flow diagram illustrating a 

methodology for developing a plan to attain and maintain a 

level of reliability and robustness (R&R) in accord with the 

present inventions- 
Figure 17a illustrates a first portion of an R&R 

checklist in accord with the present inventions- 
Figure 17b illustrates a second portion of an R&R 

checklist in accord with the present invention; 



Figure 18 is a block flow diagram illustrating an 
overview of the Optimized aspect of the present invention; 

Figure 19a is a chart demonstrating the effect of 
shifting a control factor x with respect to a linear 
transfer function y=f (x) in accord with the present 
invention; 

Figure 19b is a chart demonstrating the effect of 
shifting a control factor x with respect to a nonlinear 
transfer function y=f (x) in accord with the present 
invention; 

Figure 20a is a chart demonstrating the effect of 
shifting a control factor x where x is unaffected by system 
noises Nl and N2 in accord with the present inventions- 
Figure 20b is a chart demonstrating the effect of 
shifting a control factor x where x is affected by noise 
factors Nl and N2 in accord with the present invention; 

Figure 21a illustrates a first portion 
of a computer- implemented scorecard for iteratively 
adjusting design parameters, manufacturing tolerances, and 
noise/control factors associated with a given transfer 
function in accord with the present inventions- 
Figure 21b illustrates a second portion of a 
computer-implemented scorecard for iteratively adjusting 
design parameters, manufacturing tolerances, and 
noise/control factors associated with a given transfer 
function in accord with the present invention; and 

Figure 22 is a block flow diagram illustrating an 
overview of the verify aspect of the present invention. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Figure 1 is a block flow diagram illustrating an 
overview of a preferred method 10 for implementing the 



present invention. As represented in block 12, a "Define" 
aspect of the present invention identifies at least one 
aspect of a product that is critical to customer 
satisfaction (CTS) based on an analytical assessment of the 
"voice of the customer" 14. Preferably, the CTS aspect is 
reduced to a quantified value Y preferred by the customer 
base. "Value" Y, however, is not limited to discretely or 
numerically quantifiable values. Y may include qualitative 
descriptions, notions, or otherwise, which are identifiable 
forms or arrangements of CTS aspects which are based on the 
voice of the customer. 

As represented in block 16, a "Characterize" 
aspect of the present invention qualitatively and/or 
quantitatively characterizes each CTS aspect in terms of its 
contributing factors (y) , and characterizes each 
contributing factor y in terms of its significant control 
factors (x) and noise factors (n) . In accord with a 
preferred embodiment of the present invention, each 
contributing factor y is defined in terms of a transfer 
function y = f (x,n) . 

As represented in block 18, an "Optimize" aspect 
of the present invention minimizes variability in each y. 
In accord with a preferred embodiment of the present 
invention, variability in each y is minimized by "shifting" 
nominal x values during product design to minimize the 
response in y caused by existing manufacturing variability 

and noises n. 

In the event the necessary variability reduction 
in a particular y cannot be obtained by design optimization, 
conventional methodologies of manufacturing capability 
improvement (i.e., reduced tolerances, etc.) may be 
implemented to reduce variability in x. Notably, 
manufacturing capability improvement is often substantially 



more complex, time-consuming and expensive than design 
optimization. Accordingly, manufacturing capability 
improvement methods, as represented in block 19, should only 
be implemented as necessary after the exploitation, to the 
extent reasonable, of all opportunities for design 
optimization . 

As represented in block 20, a "Verify" aspect of 
the present invention assesses the extent to which the CTS Y 
is attained and maintained over the life of the product. 

For purposes of clarity and understanding, the 
remainder of the Detailed Description Of The Preferred 
Embodiments is divided into four principal sections, each 
corresponding to a separate aspect of the present invention 
(e.g., Define, Characterize, Optimize and Verify) . Notably, 
the present invention is described in the context of the 
automotive industry. Such illustration, explanation and 
application of the present invention, however, is not so 
limited in practice. The objectives, advantages and means 
provided in accord with the present invention may be applied 
to a myriad of products, processes and other circumstances. 

Define 

The primary objectives of the Define aspect of the 
present invention are twofold: to identify and assess the 
"voice of the customer" base for a given product (i.e., to 
understand the customer satisfaction, needs, wants and 
quality history for the product) and, based on the voice of 
the customer base, to identify CTS aspects of the product 
including the associated functional targets Y for each CTS 
aspect . 

Figure 2 is a block flow diagram illustrating an 
overview 24 of the Define aspect of the present invention. 
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Inputs 2 6 to the Define aspect include but are not limited 
to: consumer insights, quality /customer satisfaction 
history, functional product requirements, serviceability 
requirements, corporate requirements, regulatory 
requirements, system integration targets, brand profiles, 
quality loss function, quality function deployment (QFD) , 
product design specifications, business strategies, 
technology assessments, market segmentations and 
benchmarking . 

Technical activity 28 associated with the Define 
aspect of the present invention includes defining the needs 
and requirements of the product and customer, relating the 
needs and requirements to customer satisfaction and 
benchmarks, prioritizing the needs and requirements to 
prioritize the CTS aspects of the product and peer review. 

Outputs 3 0 of the Define aspect of the present 
invention include a Kano analysis and a scorecard for 
quantifying and/or qualifying the CTS aspects into 
functional target values Y. In accord with a preferred 
embodiment of the present invention, the Define scorecard 
includes a relationship between each Y and customer 
satisfaction, a benchmark of each Y among competitors 
including a best-in-class design, and acceptable targets or 
ranges Y for each identified CTS aspect. 

Consumer insight is a primary input to the Define 
aspect of the present invention. In accord with a preferred 
embodiment of the present invention, consumer insight 
includes more than simply what the customer likes and 
dislikes in a given product. More particularly, consumer 
insight includes an analysis of why a consumer likes or 
dislikes various product aspects. In addition, consumer 
insight predicts features and characteristics that will 
surprise and delight customers. In addition to 



demographics, lifestyle, product preference and usage 
habits, consumer insight considers the psychology of the 
customer to identify what emotional need is so strong it 
will cause a consumer to act. The answer to this inquiry 
provides the foundation for developing a compelling brand, 
which in turn will attract and retain a loyal customer. 

Methods for acquiring consumer insight involve the 
use of certain tools including but not limited to 
interviews, verbatims, observation, immersion (i.e., 
"stepping" into another's life for a sufficient time to gain 
insight), and introspection (i.e., imagining yourself in the 
role of the consumer) . Open-ended questions for eliciting 
consumer feedback regarding products and services includes 
but is not limited to: 

What images come to mind when you visualize 

this product or service? 

What complaints, problems or weaknesses would 
you like to mention about the product or service? 

What features do you think of when selecting 

the product or service? 

What new features might address your future 

needs? 

Organizing consumer insight data in a useful 
manner in accord with the present invention involves a 
variety of tools including but not limited to: a brand 
bull's eye, a brand profiler, a product attribute leadership 
strategy (PALS), a global quality research system (GQRS) , 
product benchmarking, a gap analysis, a Kano analysis, a 
target-setting/tradeoff analysis, a quality function 
deployment (QFD) analysis, and a CTS scorecard. 

Figure 3 illustrates an example brand bull's eye 
for organizing consumer insight pertaining to 
characteristics of a particular product (e.g., automobile). 
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The brand bull's eye helps product development teams to 
define the vision of their brand at a high or abstracted 
qualitative and/or quantitative level in terms of 
image/personality, product/service and price/value. 

Figure 4 illustrates an example brand profiler 
including "Product Attribute Leadership Strategy" (PALS) 
corresponding to the example brand bull's eye illustrated in 
Figure 3. The PALS translates the high-level bull's eye 
characteristics into actual product attributes upon which 
designers and engineers can take action. Another function 
of the PALS is to identify which product aspects or 
attributes differentiate the product from the competition. 
Table 2 contains an example listing of attribute class 
values 40 and corresponding definitions in accord with the 
present invention. 



Table 2 



Label 


Name 


Definition 


D 


Differentiator 


Attributes that are extremely 
important to differentiate 
the brand from competition. 
The Differentiator is the 
basis on which the brand's 
personality is built. 


I 


Important Brand 
Benefit 


Attributes that are an 
important part of the brand's 
personality, but do not 
differentiate the product. 


G 


Given 


Attributes whose priorities 
are determined by the cost of 
entry. Improvements are not 
critical to support the 
brand's personality. 


NA 


Not Applicable 


These attributes are not 
applicable to this brand or 
product type . 
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Table 3 contains an example qualification listing 
for product attribute leadership values 42 in accord with 
the present invention. 



Table 3 



Label 


Name 


Definition 


L 


Leadership 


Attributes that should set 
the brand apart from other 
vehicles within the 
competitive set; basis upon 
which the brand's personality 
is built. 


A 


Among The 
Leaders 


Attributes for which the 
brand must be as good as the 
best in the competitive set 
to support its positioning. 


! c 


Competitive 


Attributes for which the 
brand must be in line with 
the competitive set; but are 
not critical to the brand's 
personality. 


M 


Minimal 
Investment 


Attributes that are not 
relevant to support the 
brand; if required at all, 
should be met at the lowest 
possible cost. 


u 


Unc ompe t i t i ve 


Uncompetitive attribute. 



Sources of customer satisfaction and quality 
history data include but are not limited to satisfaction and 
market data, media ratings (e.g., JD Power, Consumer 
Reports, etc.)/ market research, historical warranty and 
recall data and benchmarking. Another source of customer 
satisfaction is a global quality research system (GQRS) . 
The GQRS includes retail customer-based evaluations through 
which customers identify and define their own perceptions of 
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vehicle quality including "things gone wrong" ( "TGW" ) , 
satisfaction, expectations and verbatim statements. Other 
metrics which may be derived from the GQRS include 
inhibitors of customer satisfaction and relative leverages. 
Relative leverage is a measure of the strength of the 
relationship between an individual satisfaction attribute 
and the satisfaction with overall product ownership 
experience. Figure 5 is a quadrant chart for relating 
Percent High Customer Satisfaction with Relative Leverage in 
accord with the present invention. 

Figure 6 is a block flow diagram illustrating an 
overview of product benchmarking in accord with the Define 
aspect of the present invention. Product benchmarking in 
accord with the present invention includes three iterative 
primary elements: planning 50, measurement /analysis 52 and 
change 54. 

Figure 7 illustrates a gap analysis in accord with 
the measurement /analysis aspect of product benchmarking. As 
illustrated, the gap analysis graphically identifies 
discrepancies between baseline and best practices for a 
plurality of product performance measures. 

The change aspect 54 of product benchmarking 
involves developing action plans that revisit the purpose of 
the benchmarking study, review the baseline product 
assessment, review external product performance metrics, 
and, based on compiled data, make recommendations to close 
some or all of the identified gaps. 

Figure 8 illustrates a Kano analysis 60 for 
analyzing data collected in accord with the Define aspect of 
the present invention. The primary objective of the Kano 
analysis is to identify the benefits and limitations of 
different options for improving customer satisfaction. In 
addition, the analysis organizes and prioritizes the CTS 
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aspects of the overall product. Ultimately, the Kano 
analysis provides a template for planning, executing and 
assessing the progress of overall product quality 
improvement. The Y-axis 62 of the Kano analysis represents 
the relative degree of customer satisfaction corresponding 
to each product attribute that is plotted. The X-axis 64 
provides an indication as to how well the desired function 
of the CTS product aspects has been achieved. 

Notably, the Kano model identifies three primary 
regions of quality, each exhibiting a different effect on 
customer satisfaction. Curve 66 represents a threshold of 
basic product quality. Basic product quality curve 66 
delineates aspects of the overall product that the customer 
only "notices" when the aspects are missing or 
malfunctioning. In other words, the customer is not more 
satisfied when these aspects perform well and optimizing 
these items will not raise customer satisfaction beyond a 
certain limit. For example, aspects of basic quality 
pertaining to an automobile may include trim defects, 
squeaks and rattles, engine malfunctions and transmission 
faults. Data sources for identifying basic quality product 
aspects include the GQRS (i.e., things gone wrong ( "TGW" ) ) 
and warranty, campaign (recall) repair and defect data. 

Performance quality curve 68 delineates aspects of 
the overall product that increase customer satisfaction the 
better they perform and decrease customer satisfaction the 
worse they perform. In accord with the automotive example, 
performance quality aspects may include fuel economy, power 
and pickup, shift quality and wind noise. Data sources for 
identifying performance quality product aspects include the 
GQRS (i.e., aspects with a high satisfaction leverage), 
attribute targets, field and benchmarking data, consumer 



interviews, and PALS aspects which are designated "Lead" or 
"Among the Leaders" . 

Excitement quality curve 70 delineates aspects of 
the overall product that increase customer satisfaction when 
the aspect is more functional, but do not decrease customer 
satisfaction when the aspect is less functional (i.e., 
surprises and delights) . In accord with the automotive 
example, excitement quality attributes may include 
improvement of existing function, new features, 
technological advances, etc. Data sources for identifying 
excitement quality aspects include the GQRS (i.e., items 
with a high percentage better than expected and customer 
verbatims), customer insight/observation (i.e., retail, 
fleet and enthusiasts), technology futuring, and PALS 
aspects which are designated "Lead" . 

Notably, excitement quality product attributes may 
vary over time. For example, aspects that are initially a 
"surprise and delight" to a customer may turn into product 
expectations over time (e.g., cup holders, new technology, 
"gadgets", etc.). Accordingly, developing "surprises and 
delights" requires activities to gain insight into 
customer's emotions and motivations, including observational 
activities that identify latent customer needs. Excitement 
quality is also driven by "superlative execution" of the 
product aspects that drive the overall product brand. 
Accordingly, a robust product design that minimizes 
degradation of these attributes over time is important to 
maintaining excitement quality. In addition, a robust 
design of a product aspect that preserves basic quality 
substantially beyond customer expectations may become an 
excitement quality aspect over time based on outstanding 
reliability and durability. 
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Figure 9 illustrates a schematic diagram (i.e., 
"House of Quality") 99 for organizing QFD data. Column 100 
contains CTS product aspects. Column 102 contains a 
prioritization of CTS aspects 100 based on a performance 
rating scale of 0 to 5 (derived from consumer surveys and 
other market data) . Row 104 contains functional product 
requirements expressed as engineering criteria. In accord 
with the automotive example, "good ride" may be defined by 
engineering criteria such as dampening, anti-roll and 
stability. Relationship matrix 106 graphically and 
quantitatively reflects the degree of relationship between 
CTS product aspects 100 and engineering criteria 104 
according to the relationship matrix key 108 where the value 
9 reflects a high degree of relationship and the value 1 
reflects a low degree of relationship. 

Row 110 contains target design values 
corresponding to each of the engineering and design criteria 
contained in row 104. Competitive assessment 112 
graphically reflects a comparison, based on consumer data, 
between current CTS product aspects 100 and the same or 
similar aspects of competitors' products. Similarly, 
competitive assessment 114 graphically reflects a 
comparison, based on engineering and technical data, between 
current engineering and design criteria 104 and the same or 
similar criteria of competitors' products. 

Correlation matrix 118 facilitates tradeoff 
decisions by correlating between various engineering 
criteria 104 and qualifying the level of correlation (i.e., 
the relative impact each metric has on the other) . 

Technical importance ratings contained in row 120 
are derived for each engineering/design criteria 104 based 
on the cumulative sum of the products of the relevant 
customer importance ratings 102 and the relevant 
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relationship matrix values 106. For example, the technical 
importance rating for engineering/design criteria "a" is 
calculated as follows: (5x9) + (4x9) = 63. Similarly, 
the technical importance rating for engineering/design 
criteria "d" is calculated as follows: (3x3) + (5x1) = 
14. 

Notably, the House of Quality 99 may be 
rearranged, modified or adapted within the scope of the 
present invention to best fit the particular circumstances 
unique to any QFD analysis. In addition, not all aspects of 
the House of Quality 99 are required for an effective QFD 
analysis. The necessary elements depend uniquely upon the 
circumstances of each QFD analysis. 

Figure 10 illustrates a scorecard for assessing 
and summarizing the CTS aspects of an overall product that 
are identified in accord with the Define aspect of the 
present invention. CTS or surrogate field 140 denotes the 
aspects of the overall product that are most critical to 
customer satisfaction. Units field 142 denotes the unit of 
measure for the CTS aspect. "T.F. Y/N" field 144 denotes 
whether a transfer function (T/F) for the corresponding CTS 
aspect is currently known. Transfer functions are discussed 
in more detail with respect to the Characterize and Optimize 
aspect of the present invention. 

Status fields 145 denote the current mean (a) 146a 
and variability (ay) 146b for the CTS aspect. 
Competitor/Best-in-Class (BIC) fields 148 denote a 
competitive benchmark of the mean 148a and variability 148b 
values for the corresponding CTS aspect 140. 

Target Initial Condition fields 150 denote the 
target mean 150a (i.e., Y value), lower specification limit 
(LSD 150b and upper specification limit (US) 150c values 
for the CTS aspect shortly after manufacture or assembly of 
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the overall product. Target aged condition fields 152 
denote the target mean 152a, LSL 152b and US 152c values for 
the CTS aspect in an aged condition. 

"Associated Customer Satisfaction Improvement" 
field 154 denotes an expected level of improvement in 
customer satisfaction as a result of attaining the target 
CTS aspect values denoted in fields 150 and 152. 

Characterize 

The primary objectives of the Characterize aspect 
of the present invention are threefold: to break down each 
CTS aspect into its contributing factors (y) ; to 
characterize each y in terms of its significant control 
factors (x) and noise factors (n) ; and to develop a transfer 
function y=f (x,n) which quantitatively and/or qualitatively 
relates CTS contributing factors y with their respective 
control factors x and noise factors n. 

Table 4 contains nomenclature and corresponding 
descriptions utilized in accord with the present invention. 



Table 4 



Symbol 


Description 


Y 


Customer Need/Want that is critical to 
Satisfaction (CTS) . 

If not expressible as a measurable technical 
metric, can be represented by surrogate measures 
that are. 


\r 

y 


Measurable quantity that is critical to 
achievincr Y. 

Not directly specifiable by an engineer. 


X 


Measurable quantity that is critical to 
achieving y. 

Can be specified by an engineer in the 
product or process design, e.g., part and process 
characteristics 

* Control Factor." 


n 


Measurable quantity that affects y 
Cannot be specified by an engineer 
"Noise Factor . " 



Figure 11 is a block flow diagram illustrating an 
overview 160 of the Characterize aspect of the present 
invention. Inputs 162 include but are not limited to: the 
Kano diagram, the CTS aspects including target and target 
range values (i.e., Y's) for corresponding 
engineering/design criteria derived from the QFD as 
(illustrated in Figure 9), the CTS scorecard (illustrated in 
Figure 10), functional system boundaries and interfaces, and 
existing process and hardware failure mode and effects 
analysis (FLEA/PFMEA) data. 

Technical activity 164 associated with the 
Characterize aspect of the present invention includes 
decomposing each CTS Y into its contributing factors y, and 
further decomposing each y into its contributing control 
factors x and n noise factors. Additional activities 
associated with the Characterize aspect of the present 



invention include generating a quantitative and/or 
qualitative function structure to create a general or 
explicit transfer function y=f (x,n) and peer review. 

Outputs 166 of the Characterize aspect of the 
present invention include function diagrams, a graphical map 
correlating Y 1# y if Xi and m values, a transfer function for 
each yi, a scorecard with target and range values for each y± 
and xi, and a preliminary plan for the Optimization and 
Verify aspects of the present invention. 

Figure 12a illustrates a function map template for 
characterizing CTS aspects in accord with the present 
invention. Figure 12b illustrates various methodologies and 
tools that may be implemented to understand the product or 
system at issue and to generate the corresponding transfer 
function y=f (x,n) . 

In accord with a preferred embodiment of the 
present invention, function modeling provides the starting 
point for the systematic relational characterization of Y if 
y i7 xi and n x values. In more complex systems, the modeling 
function helps to identify and quantify /qualify the CTS 
aspects of a product as well as the applicable transfer 
function(s) . Generally, the function model is a form- 
independent statement of what a product or statement does. 
Modeling functions are usually expressed as a verb-noun pair 
or as an active verb-noun phrase pair. General examples 
include "transmit energy", "import material" and "indicate 
state". More specific examples include "transmit torque", 
"import hand" and "indicate temperature". 

An overall function model is a statement of what 
the entire product, system or process does. A subfunction 
is a component function which is combined with other sub- 
functions to create the overall product function. For 
example, the overall function of a coffee cup is to "hold 



coffee". Sub-functions include "contain liquid", "insulate 
air", "support liquid", "accept pour", "interface hand" and 
"pour liquid" . Notably, some CTS product aspects deal with 
metrics that do not define function (or subfunction) 
directly (e.g., cost, reliability, weight, appearance, 
productability, environmental impact, etc.). 

In accord with conventional notions of function 
modeling, constraints are properties of a product or system, 
but not something the product or system does. Typically, 
all elements in a given product or system contribute to a 
constraint — not just one element. In addition, 
constraints can be modeled with metrics over the function 
model or over the bill of materials. 

A function structure is used to validate a 
functional model. Figure 13 is a block flow diagram 
illustrating a preferred method for creating a function 
structure 174 in accord with the present invention. 

First, an overall function model for the product, 
system or portfolio is created, as represented in block 176. 
Next, an activity diagram is developed which includes a 
system boundary and user activities as represented in block 
178. Next, CTS aspects are mapped to input flows as 
represented in block 180. For each input flow, a function 
chain is created from input to output as represented in 
block 182. As represented in block 184, each function chain 
is aggregated into an overall function structure. Next, the 
functional decomposition of the functional structure is 
validated as represented in block 186. Last, the overall 
function model is verified against each CTS aspect, as 
represented in block 188. 

Identifying technical metrics or 
engineering /design criteria in accord with the present 
invention comprises a three-tiered process. First, each 



critical product or system function designated in the QFD is 
measured to quantify the function as a "y" . Next, 
interfaces between each critical function designated in the 
QFD are measured to quantify the conflict/reinforcement as a 
»y". Last, all possible "y's" are rolled into an effective 
set of "y's" . 

To measure each critical system function, the 
function model is analyzed to determine what value can be 
measured to quantify the transformed function flow from 
input to output. The results of this inquiry are potential 
"y's". This approach to measuring system functions provides 
the input-output relationship explicitly in a metric-based 
manner. Preferably, system functions are laid out 
geometrically so that separate functions are contained 
within separate regions. Figure 14 illustrates an example 
geometric function structure for an automotive regenerative 
braking system containing a variety of separate functions 
each having inputs, transformations and outputs. 

For each system interface having a high level of 
interactivity, three inquiries are made: the level of flow 
between the two functions, the problem/benefit associated 
with the flow, and what can be measured to quantify this 
flow problem/benefit. In accord with the coffee example, 
the flow between "contain coffee" and "guide pour" is 
"coffee". This coffee flow causes a benefit /problem because 
the "guide pour" function must provide an opening that is 
disadvantageous to "contain coffee". The focus of the 
difficulty/benefit interaction in this case is the sloshing 
of coffee during pour-out. Accordingly, measurable values 
constituting potential "y" values include the amount of 
slosh from side force impact during pour and the amount of 
spill during a 15E pour from a full cup. 



Preferably, the QFD is updated to include any new 
"y" values (i.e., engineering/design metrics or criteria 
contained in Figure 9, row 104). In addition, the CTS 
scorecard illustrated in Figure 10 is updated to include any 
additional "y" values identified throughout the Characterize 
and Optimize aspects of the present invention. 

The function mapping aspect of the present 
invention involves identifying the control and noise 
factors, x and n respectively, contributing to each y. To 
identify x values, function input variables are examined to 
determine which variables can be readily specified by design 
or manufacturing teams. Any variables that can be 
controlled are considered control factors x. Any variables 
that cannot be controlled are considered noise factors n. 

Figure 15 illustrates transfer function format 190 
and example transfer functions 192 for quantitatively 
interrelating Y 1# y ir xi and n± values in accord with the 
present invention. The primary purpose of the transfer 
function is to estimate both the mean and variance of CTS 
product aspects (i.e., Y values) as well as their 
contributing elements y. Ultimately, the transfer function 
is utilized to determine for what values of the independent 
variable the dependent variable is a maximum, minimum, or 
specified value with minimum variability. 

Transfer functions may be derived based on a wide 
variety of information ranging from engineering "first 
principles" to more empirical data. For example, one source 
of transfer functions includes technical drawings of 
components and systems. Commonly, such drawings are useful 
to determining product variation stack-up based upon 
tolerances and geometry. Another source of transfer 
functions includes physical equations that describe function 
(e.g., v=IR, f=ma, etc.). A third source of transfer 



functions includes finite element and other virtual product 
models. Generally, such models are derived from first 
principles but are not expressible in a single closed-form 
transfer function. A fourth source of transfer functions 
includes designed experiments based on models, prototypes, 
or representative hardware. Yet another source of transfer 
functions is a correlation/regression analysis based on 
product, manufacturing, or service performance data. 

In some circumstances, a CTS product aspect cannot 
be expressed through an explicit transfer function. In this 
case, a set of metrics is identified that can serve as a 
surrogate for Y. Additional data is generally required in 
this situation to verify the relationship between Y and the 
surrogate metrics . 

Notably, not all x and n factors should be 
included in the overall transfer function. Preferably, only 
the x and n factors that are most critical for achieving y 
are included. For example, only a few (i.e., two or three) 
x factors may contribute to 80% of the variation in a given 
y. While over thirty x factors may contribute to the 
remaining 20% of the variation in y. Preferably, in this 
case, the transfer function includes the few x values that 
contribute to 80% of the variation in y. 

Characterizing opportunities for increasing 
product robustness in accord with the Characterize aspect of 
the present invention also involves the implementation of a 
Pugh methodology for identifying, defining and prioritizing 
favorable design improvements. The Pugh methodology is 
advantageous because it facilitates concept generation in a 
structural manner and typically produces a multitude of 
concepts that must be objectively evaluated. In addition, 
the methodology improves original concepts, generates hybrid 
concepts, and increases the efficiency of development time 



and money. Due to its structural nature, the Pugh 
methodology iteratively focuses the design team on a defined 
set of alternatives, and prevents the team from prematurely 
jumping to a single solution that undergoes frequent 
engineering changes. In addition, the Pugh methodology 
increases the understanding the engineering and design teams 
have of overall product design. 

In general, the Pugh methodology evaluates design 
concepts against design criteria at a relatively low 
resolution. The method employs an iterative process of 
design alternative selection and enhancement to "converge" 
on the best overall architecture. Table 5 contains a 
generic Pugh analysis in accord with the present invention. 
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To begin the Pugh analysis, a design concept (not 
shown) is selected as a reference against which all 
potential design alternatives will be compared. Each 
alternate design concept (i.e., A, B, C, etc.) or hybrid 
concept (i.e., BC) is then evaluated and benchmarked against 
the reference design based on the expected performance of 
the alternative. For each design alternative, a design 
criteria that is expected to perform better than the 
reference design is denoted with a " + " . Criteria that are 
expected to perform worse than the reference design were 
denoted with a Design criteria that are expected to 

perform the same as the reference design are noted with an 
"S" . When all design alternatives and corresponding 
criteria have been benchmarked, the respective number of 
" + , s" , "-'s" and "S's" are summed. In a single round of 
analysis, the winning concept is the one with the fewest 
»-' S », in the case of a tie, the winning concept is the one 
with the most "+ , s" in addition to the fewest "-'s". 
Because the Pugh methodology is an iterative process, 
alternate potential concepts may be improved or refined to 
eliminate "-'s n . 

Figure 16 is a block flow diagram illustrating a 
methodology 270 for developing a plan to attain and maintain 
a level of reliability and robustness (R&R) in accord with 
the Characterize aspect of the present invention. Figures 
17a and 17b illustrate an (R&R) checklist for quantitatively 
and/or qualitatively defining and assessing the R&R plan. 
Notably, various aspects of the R&R planning methodology are 
addressed in greater detail with respect to the Optimize and 
Verify aspects of the present invention. 

In accord with a preferred embodiment of the 
present invention, the R&R planning methodology begins, as 
represented in Figure 16 (block 274), by listing all of the 



CTS aspects identified during the Define phase within region 
298 of the R&R checklist shown in Figure 17a. Next, as 
represented in Figure 16 (block 276), potential error states 
and failure modes are identified for the overall 
5 system/subsystem/component and listed in region 3 00 of the 

R&R checklist. This data may be readily attained from 
various sources including FLEA/ PFMEA data and other 
engineering data. 

As represented in Figure 16, block 278, the 
10 control factors x included in the corresponding transfer 

function are listed in region 3 02 of the R&R checklist. 
Next, any relationships between the listed CTS aspects and 
control factors are denoted in the R&R checklist matrix area 
304. As represented in block 280, potential noise factors n 
15 are identified and listed in region 306 (Figure 17b) of the 

R&R checklist. Although the R&R checklist illustrated in 
Figure 17b contains five example noise categories, any 
variety or number of noise categories may be provided to 
best suit the particular product, system or process at 
20 issue. 

Next, the failure mode/noise interaction matrix is 
generated in R&R checklist region 308, as represented in 
Figure 16, block 282. Within the matrix, any interactions 
between failure modes and noise factors are specified to 

25 identify potential mechanisms of failure. 

As represented in Figure 16, block 284, a noise 
factor management strategy is initiated for key 
characteristics controlled by manufacturing processes (i.e., 
Noise 1). Notably, output from the Optimization aspect of 

30 the present invention is necessary before the noise 

management strategy can be initiated for the remaining noise 
categories (i.e., Noise 2 through Noise 5). Example noise 
factor management strategies in accord with the present 
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invention include but are not limited to: changing existing 
technology, applying parameter design, upgrading current 
design specifications, reducing or removing noise, adding a 
noise compensation device, and disguising or diverting the 
5 noise. Tools for implementing noise factor management are 

well known and include but are not limited to process 
failure mode analyses (PFMA) , process checks, statistical 
process control (SPC) , and control planning. 

As represented in Figure 16, block 288, a 
10 potential test strategy is defined for measuring and 

evaluating the functional requirements (i.e., CTS Y's). In 
most cases, information for performing test strategies may 
be captured from corresponding design verification planning 
reports (DVPR) , and input into R&R checklist region 312. 
15 Next, any relationship between the test strategies 

(region 312) and the CTS aspects (region 298) is indicated 
in R&R checklist region 314, as represented in Figure 16, 
block 290. Notably, any obvious "holes" or gaps identified 
in regions 314 or 318 should be addressed to ensure that at 
20 least one test strategy 312 exists for each CTS aspect 298 

and noise 3 06, respectively. 

Optimize 

25 The Optimize aspect of the present invention 

includes several objectives. One objective is to understand 
the capability and stability of current manufacturing 
processes contributing to the overall product. Another 
objective is to understand and optimize the current 

30 robustness of the product itself. A third objective is to 

minimize product sensitivity to manufacturing noise. Yet 
another objective of the Optimize aspect is to minimize 
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manufacturing process sensitivity to product and 
manufacturing variations . 

Figure 18 is a block flow diagram illustrating an 
overview 340 of the Optimize aspect of the present 
invention. Inputs 342 include but are not limited to: 
present manufacturing process capabilities including <J> 
(mean), target Y and ay (variability) values; a P-Diagram 
including critical y, x and n criteria; an updated transfer 
function (as developed to date) ; manufacturing and assembly 
process flow diagrams and maps; robustness and reliability 
capability studies; PFMEA and DFMEA data; and verification 
plans including the R&R checklist and associated 
manufacturing noise management strategies . 

Technical activity 344 associated with the 
Optimize aspect of the present invention includes minimizing 
variability in y values by selecting nominal x values, 
optimizing manufacturing processes as necessary to achieve 
the highest ay value possible, ensuring the ease of assembly 
and manufacturability of the product, eliminating specific 
failure modes, updating the overall control plan, and 
conducting a peer review. 

Outputs 346 of the Optimize aspect of the present 
invention include an updated transfer function, a scorecard 
including an estimate of a y , target nominal x values, a 
variability metric for each CTS Y and related functions 
(e.g., range, standard deviation, S/N ratio improvement, 
etc.), tolerances specified for important product 
characteristics, a short-term Z~score capability, long-term 
product capabilities, updated verification plans including 
an updated R&R checklist, and an updated control plan. 

More particularly, the Optimization aspect of the 
present invention seeks to identify design parameter 
settings or nominals x that, for each transfer function y 



achieve a target response in y with a minimum level of 
variability despite noises n, while satisfying all 
applicable system constraints. 

Notably, transfer functions f(y) generated in 
accord with the present invention typically fall into two 
categories: linear and nonlinear. The impact that design 
parameters x have on the response in y varies greatly 
depending on the order of the corresponding transfer 
function. Consider the generic transfer functions y = f (x) 
plotted in Figures 19a and 19b. Figure 19a illustrates a 
plot of a linear transfer function y = f (x) where the x-axis 
351 reflects variability 350 associated with a control 
factor x and the y-axis 3 53 represents the resulting 
response or variability 3 52 in y due to the position of x. 
Because the transfer function f (x) plotted in Figure 19a is 
linear, shifting the nominal value of x (e.g., from x a to x b ) 
does not impact the response 352 in y where variability 3 50 
in x (e.g., manufacturing variability, etc.) remains 
constant . 

In contrast, Figure 19b demonstrates that with a 
nonlinear transfer function f (x) , the response in y varies 
substantially depending upon the nominal value of control 
factor x where variability 350 in x remains constant. By 
adjusting the nominal value of x from x a to x b , the 
variability experienced in y effectively "shrinks" from 3 54 
to 356 based on the nominal position of x with respect to 
f (x) . 

Accordingly, where a transfer function generated 
in accord with the present invention is nonlinear, the 
nominal values of x may be optimized in the design phase of 
product development to desensitize the response in y to x 
variability, therefore yielding a more robust product 
design. To compensate for changes in nominal y values due 



to optimization of design parameters x contributing to 
nonlinear transfer functions, other y nominal s that are 
based on linear transfer functions may be shifted without 
sacrificing response variability, as shown in Figure 19a, to 
bring about the desired CTS Y with a minimum level of 
variability. 

In the event that the target response in y or the 
target CTS Y cannot be obtained based on the optimization of 
nonlinear transfer function design parameters x and the 
shifting of linear transfer function design parameters x to 
compensate, conventional techniques for optimizing 
manufacturing processes may be implemented. One such 
technique involves the reduction of variability in x by 
decreasing manufacturing tolerances. Notably, the reduction 
of variability in x through the optimization of 
manufacturing processes is often substantially more 
expensive than optimizing product design parameters prior to 
the product being manufactured. Accordingly, attempts to 
reduce variability in x should take place only after nominal 
design parameters x contributing to their respective 
transfer functions are optimized to their full potential. 

Other techniques for increasing the reliability 
and robustness of an overall product include physically 
strengthening the existing design or changing the overall 
design concept altogether. Notably, a fundamental change in 
the overall design concept of a product will typically 
result in a new transfer function having unique design and 
noise parameters x and n respectively. 

Referring now to Figures 2 0a and 20b, the effect 
noise n has on the response in y for a given x is shown. In 
Figure 20a control factor x is unaffected by the combined 
impact of noises Ni and N 2 . Accordingly, shifting control 
factor x from x a to x b has no impact on the variability 3 58 



experienced in y. In contrast, where the control factor x 
interacts with noise Ni and N 2 , as shown in Figure 20b, 
adjusting (optimizing) the control factor x nominal from x b 
to x a substantially * shrinks " or desensitizes the response in 
y and, therefore, yields a more robust product design. 

In accord with a preferred embodiment of the 
present invention, variability (o y ) experienced in y based on 
a transfer function y=f (x) is defined according to Equation 
1. 



Variability of y = cr y = 
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Referring to Equation 1, the value 



K dx l} 



represents sensitivity or variability in y. For linear 



transfer functions, 



dx 



transfer functions, however, 



is constant. For nonlinear 



varies and can, therefore, 



V dx) 

be optimized during product design and development, as 
illustrated and described with respect to Figures 19 and 20. 
Referring again to Equation 1, the value a 2 xl represents 
variability in y due to manufacturing process variability in 
x. 

In manufacturing situations involving the assembly 
of various mechanical components, a statistical tolerancing 
methodology may be implemented to develop and/or improve 
upon an existing design concept and corresponding transfer 
function. The statistical tolerancing methodology primarily 
focuses on optimizing the sequence of component assembly, 



the degrees of freedom associated with assembly and design 
parameter nominals and tolerances to identify the most 
appropriate aspects of an overall product to optimize. Put 
another way, statistical tolerancing analyzes existing 
manufacturing or product constraints to pinpoint the most 
effective or cost-efficient opportunities to optimize y in 
accord with the present invention. For example, a given CTS 
Y embodied in a physical mechanical assembly may have 
several contributing sources of variability a y . By simply 
adjusting the order of assembly, nominal x parameters, 
manufacturing tolerances and degrees of freedom associated 
with the design and assembly, some sources of variability in 
y might be identified whose optimization may be more cost- 
effective or have a more significant impact on the target 
and desired variability for the overall CTS Y as compared to 
other sources of variability. Accordingly, statistical 
tolerancing effectively prioritizes opportunities for 
increasing robustness in a given mechanical assembly. 

Methodologies for implementing statistical 
tolerancing in accord with the present invention include 
techniques such as Worst Case Analyses and Monte Carlo 
Simulation which are well known and understood by those of 
ordinary skill in the field of manufacturing optimization 
methodologies . 

Figures 21a and 21b illustrate a computer- 
implemented scorecard 370 for iteratively adjusting design 
parameters, manufacturing tolerances and noise/control 
factors associated with a given transfer function y=f (x,n) 
to approach a target y value while minimizing variability in 
y to an acceptable level. In accord with a preferred 
embodiment of the present invention, the scorecard 37 0 is 
implemented in a spreadsheet application having function 



solver capabilities. Suitable applications include 
Microsoft Excel, Borland Quatro-Pro and Lotus 1-2-3. 

In cell 372, a user inputs a description of a y 
value currently being optimized (e.g., compression ratio). 
In cell 374, the user inputs the unit of measure for the y 
value (e.g., inches, centimeters, ratio, etc.). In cell 
376, the user inputs the transfer function y=f (x,n) 
corresponding to the characteristic 372. In specification 
fields 378, the user inputs a target nominal value O, an 
upper specification limit (US) and a lower specification 
limit (LSD for the characteristic. 

In variables columns 380, the user inputs a 
description of control factors x and noise factors n 
contributing to the target y value (i.e., x and n variables 
included within transfer function 376), and a unit of 
measure for each x and n factor. In range columns 382, the 
user inputs the broadest possible range for each x and n 
factor that the current product design concept can 
accommodate. In specification columns 3 84, the user inputs, 
for each x, a lower specification limit and an upper 
specification limit corresponding to a desired mean 
(nominal) value a for x. 

In sample/database statistics columns 3 86, the 
user inputs a nominal value a and standard deviation ay for 
each x and n factor. 

In accord with a preferred embodiment of the 
present invention, a function solver utility automatically 
calculates a predicted performance capability 388 (i.e., a, 
ay, etc.) for the product characteristic 372 based on the 
transfer function 376, and the user-specified a and ay 
values for each x and n parameter. In accord with a 
preferred embodiment of the present invention, Equation 1, 
or a derivative thereof, is programmatically solved by the 



function solver based on the transfer function 376 and user- 
input a and ay values 386 to predict an overall variability 
cy 390 in characteristic 372. 

In further accord with a preferred embodiment of 
the present invention, the function solver utility is 
configured to calculate the relative percent that each 
factor x and n contributes to the overall variability in the 
product characteristic 372. Based on the contribution to 
variability each x and n factor has on overall product 
characteristic y, transfer function 376 may be modified to 
eliminate factors that have an insignificant impact on 
variability, as the expense for controls to measure and 
minimize such factors may not be cost-effective given their 

low impact level. 

A systematic or iterative process of adjusting a 
and ay for each control factor x is implemented to attain 
the most favorable a 392 and ay 390 for characteristic 372. 
It is additionally preferred that the a nominal design 
values for each control factor x are iteratively adjusted 
prior to adjusting the corresponding ay valves which reflect 
current manufacturing capabilities for each factor. 
Optimizing design parameters prior to increasing current 
manufacturing capabilities in this manner takes full 
advantage of relatively "free" product design opportunities 
before investing valuable time and resources to make more 
expensive improvements in current manufacturing 
capabilities . 

Verify 

The primary objectives of the Verify aspect of the 
present invention are twofold: to assess actual 
performance, reliability and manufacturing capabilities of 
the product after Optimization has taken place, and to 



demonstrate consumer-correlated performance over the 
product's life span. 

Figure 22 is a block flow diagram illustrating an 
overview 400 of the Verify aspect of the present invention. 
Inputs 402 to the Verify aspect include but are not limited 
to an updated verification plan including an updated R&R 
checklist, the scorecard illustrated in Figure 21 with 
predicted values of y and a y based on u x and a x , a historical 
DVP&R and a product and manufacturing process control plan. 

Technical activity 404 associated with the Verify 
aspect of the present invention includes conducting physical 
and analytical performance and key life tests, enhancing the 
tests with key noise factors, improving the ability of the 
tests to discriminate between good and bad commodities, 
applying a test strategy to maximize resource efficiency, 

and peer review. 

Outputs 406 associated with the Verify aspect of 
the present invention include an assessment of product 
performance over time including long term robustness 
capabilities, a completed R&R checklist including a 
demonstration matrix, a score card with actual y and o y 
values and lessons learned. 

While the best mode for carrying out the invention 
has been described in detail, those familiar with the art to 
which this invention relates will recognize various 
alternative designs and embodiments for practicing the 
invention as defined by the following claims. 



